Segmentation based Multi-View Stereo

This paper presents a segmentation based multiview stereo reconstruction method. We address (i) dealing with uninformative texture in very homogeneous image areas and (ii) processing of large images in affordable time. To avoid searching for optimal surface position and orientation based on uninformative texture, we (over)segment images into segments of low variation of color and intensity and use each segment to generate a candidate 3D planar patch explaining the underlying 3D surface. Every point of the surface is explained by multiple candidate patches generated from image segments from different images. Observing that the correctly reconstructed surface is consistently generated from different images, the candidates that do not have consistent support by other candidates from other images are rejected. This approach leads to stable and good results since (i) we use larger 3D patches in homogeneous image areas where small patches covered by uninformative texture would lead to ambiguous results, and (ii) we accept only candidates that are consistent across several images. Since the image segmentation used is very fast and it considerably reduces the number of candidates per image on typical scenes, we typically generate and test relatively small number of 3D hypotheses per image and thus can process large images in affordable time. We demonstrate the performance of our algorithm on large images from Strecha’s dataset.